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Abstract 

Bae et al. [6] consider the problem of optimal control of a finite dam using 
P^ r policies, assuming that the input process is a compound Poisson process 
with a negative drift. Lam and Lou [8] treat the case where the input is a Wiener 
process with a reflecting boundary at its infimum, with drift term /i > 0, using 
the long-run average and total discounted cost criteria. Attia [4] obtains results 
similar to those of Lam and Lou, through simpler and more direct methods. 
Zuckermann [f 2] considers policies when the input process is a is Wiener 
process with drift term // > 0. The techniques used by the above mentioned 
authors involve solving systems of differential or integral equations. In this 
paper we use the theory and methods of scale functions of Levy processes to 
unify and extend the results of these authors. 
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1. Introduction and summary 



Suppose that a dam has capacity V. Its water input / = (It,t > 0), is 
assumed to be a Levy process with drift /i, variance a 2 , and the water is released 
at one of two rates or M units per unit of time. We consider Pjf T policies 
in which the water release rate is assumed to be zero until the water crosses 
level A, (0 < A < V), when the water is released at rate M until it reaches 
level r, (0 < r < A). Once level r is reached, the release rate remains zero 
until level A is reached again, and the cycle is repeated. We deal with the 
cases where the input process is spectrally positive Levy, and spectrally positive 
Levy reflected at its infimum. In both cases the content process is a delayed 
regenerative process with regeneration points being the times of successive visits 
to state t. During a given cycle, the dam's water content is a Levy process with 
coefficients /i and cr 2 , and it remains so until it crosses level A; from then until 
it drops to level r again the content level behaves like a Levy process reflected 



f 



at V with coefficients [i* = n — M, er 2 , denoted by I* = (7 t *,t > 0). At any 
time, the release rate can be increased from to M with a starting cost KiM, 
or decreased from M to zero with a closing cost K 2 M. Moreover, for each unit 
of output, a reward R is received. Furthermore, there is a penalty cost which 
accrues at a rate / , where / a bounded measurable function. For the first case 
we extend the results of Zuckcrman [12] who assumed that r = and / = 0. 
Our results in the second case extend the results of Lam and Lou [8] and Attia 
[4] , where they assumed that the input process is a Wiener process reflected at 
its infimum. They also extend those of Bae et al. [6], who consider the case 
where the water input is a compound Poisson process with negative drift. Lee 
and Ahn [9] consider the long-run average cost case, for the P£* policy, when 
the water input is a compound Poisson process. Abdel-Hameed [1] treats the 
case where the water input is a compound Poisson process with a positive drift. 
He obtains the total discounted as well as the long-run average costs. Bae et 
al. [5 ] consider the P*q policy in assessing the workload of an M/G/l queuing 
system. The techniques used by in [12], [8], [4], and [6] involve solving systems 
of differential or integral equations. In this paper we use the theory and methods 
of scale functions of Levy processes, an approach not uses by researchers in this 
area before. 

In Section 2 we define the input processes and discuss their properties. In 
Section 3 we obtain formulas needed for computing the cost functionals. In 
Section 4, we discuss the cost functionals using the total discounted as well 
as the long-run average cost cases. In section 5 we discuss the special cases 
where the input process is a Gaussian process, a Gaussian process reflected at 
its infimum and a spectrally positive Levy process of bounded variation. 



2. Spectrally positive Levy processes and scale func- 
tions 

In this section we give some basic definitions; describe spectrally positive 
Levy processes and discuss some of their characteristics. The reader is referred 
to [7] for a more detailed discussion of the definitions and results mentioned in 
this section. 

For any process Y = {Y t ,t > 0} with state space E, any Borel set A 
C E and any functional /, E y (f) denotes the expectation of / conditional on 
Yo = y, P y (A) denotes the corresponding probability measure and Ia( ) is the 
indicator function of the set A. In the sequel we will write indifferently Po or 
P and E Q or E. Throughout, we let R = (-00,00), R + = [0,oo), N = {1,2,...} 
and N + = {0, 1, ...}. For x,y € R, we define xVy = xmax y and xAy — xminy. 

For every t > 0, we define Y±_ = inf (Y s ,A0), Y t = sup (Y s , V0). 

0<s<t 0<s<t 
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We will use the term "increasing" to mean "non-decreasing" throughout this 
paper. 

Definition 1. A Levy process L — {L t ,t > 0} with state space R is said 
to be spectrally positive Levy process, if it has no negative jumps. 

It follows that, for each 9 e R + ,x € R, 

E[e -e L t] = e t m ^ 

where 

nl 2 /-oo 

<j>{9) = -aB + — - j (1 - e~ 6x - 6xl {x<1} )v(dx). (2.1) 

The terms a € R, a 2 £ R+ are the drift and variance of the spectrally positive 
Levy process, respectively. The Levy measure v is a positive measure on (0, oo) 
satisfying J^°(x 2 A l)v(dx) < oo. 

The function <j> is known as the Levy exponent, and it is strictly convex and 
tends to infinity as 9 tends to infinity. For a e R + , we define 

r}(a) = sup{9 : <j>(9) = a} (2.2), 

the largest root of the equation <j>(6) = a. It is seen that this equation has at 
most two roots, one of which is the zero root. Note that, E{L\ ) = xv(dx) + 
\x. Furthermore, lim L t — oo if and only if E(L\ ) > 0, and lim L t — — oo if 

t— >-oo t— yoo 

and only if E(L\ ) < 0. Also, if E(L\ ) = 0, then lim L t does not exist. 

t— >oo 

Furthermore, 77(0) > 0, if and only if E{L X ) > 0. 

An important case is when the process L is of bounded variations, i.e., a 2 = 
and f£°(xA l)v{dx) < 00. Let 

( = —a + / xv(dx). 
Jo 

In this case we can write 

/>oo 

</>(6») =(9- (1 - e- 6x )v(dx), (2.3) 
Jo 

where necessarily £ is strictly positive. 

Definition 2. A Levy process is said to be spectrally negative if it has no 
positive jumps. 

A 

Definition 3. For any spectrally positive Levy input process L, we let L 
= - L . 
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A 

It is clear that L is spectrally positive if and only if the process L is 
spectrally negative. 

We now introduce tools, which will be central in the rest of this paper. 

Definition 4. For any spectrally positive Levy process with Levy exponent 
<fi and for a > 0, the a— scale function W a : R -» R + , W a (x) = for every 
x < 0, and on [0, oo) it is defined as the unique right continuous increasing 
function such that 

e -P x W (a) (x)dx = — -i ,0>v(a) ( 2 -4) 

<P(P) ~ at 

We will denote W° by W throughout. For a > 0, we have (see (8.24) of [ 7]) 

oo 

W {a \x) = a k W* {k+1 \x), (2.5) 

fc=0 

where W*^ is the kth convolution of W with itself. 

It follows that (0+) = if and only if the process L is of unbounded vari- 
ation. Furthermore, is right and left differentiable on (0, oo). By (x), 
we will denote the right derivative of in x. 

The adjoint a— scale function associated with (denoted by Z^) is 

defined as follows: 

Definition 5. For a > 0, the adjoint a— scale : R —> [l,oo) is defined 

as 

Z (a \x) = l + a [ W a (y)dy. (2.6) 
Jo 

It follows that as x -> oo, for a > 0, (x) ~ f,^\ , and SS x r \ ~ -r^. 

' ' y ' 4> (77(a) ) Wy<*>(x) r\(a) 

3. Basic results 

For each t e R + , let Z t be the dam content at time t , Z = {Z t ,t € R+}- We 
define the following sequence of stopping times : 

T = M{t>0:Z t >\}, To = inf{t > T : Z t = r}, 

T„ = inf{i > T n _i : Z t > A}, T„ = inf{i > T n :Z t = T },n = 1, 2, ... (3.1) 

It follows that the process Z is a delayed regenerative process with regeneration 
points {T„, n = 0, 1, ...}. 



L 
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We define the bivariate process B = (Z,R), where for t > Rt is the 
release rate (0 or M) at time t. The process B has as its state space the pair of 
line segments 



S=[(1,X)x{0}]u[(t,V] x{M}}, 
where / is the lower bound of the state space of the input process /. 

The penalty cost rate function is given by 

f(,\-\ (z,r)£{l,X)x{0} 
nz) ~\ g*(z), (z,r)e(r,V]x{M} 

where g : (I, A) — > R + and g* : (r, V] — > R + arc bounded measurable function. 
For a G R + , let the C a (x, A, 0) and C a (x, r, M) be the expected discounted 

A A * 

penalty costs during the interval [0, T ), and during the interval [To, T ) starting 
at x, respectively. It follows that, for x e [t,V] 

A 

C a (x,X,0) = E x f ° e- at g(I t )dt, 
Jo 

and for x € [A, V] 

C Q (ar,r,M) = ^ f T " e- at g*(i:)dt. (3.3) 
Jo 

The functionals (3.3), £ x [e- aT °], ^[fo], E x [e- aT °], ^[To], which we aim 
to evalautc, arc needed to obtain the total discounted and the long-run average 
costs associated with the Pjf T policy, discussed in Section 4. 

For any a e R, we define T+ = mi{t > : I t > a}, T~ = mi{t > : I t < 

A A 

a}, T+ = inf{i > : I t > a} and T" = inf{i > : I t < a}. We note that 

A 

To = almost everywhere. 

To derive C a (x, A, 0), E x [e~ a °], S x [To] we define the process obtained by 

A 

killing the process J at To, as follows: 

X t = {I t ,t<T }. (3.4) 

It is known that this killed process is a strong Markov process, with state space 
(I, A). 
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For any Borel set A C (I, A), and t € R+, the probability transition function 
of this process is given as follows 

P t (x,A) = P x (I t e A,t <T ) 
and for each a € i?+ its a— potential is defined as follows 



poo pTq 

U a (x,A) = P t (x,A)e- at dt = E x e - at I {heA} dt. (3.5) 
Jo Jo 

We note that for x < A 

C Q (a;,A,0) = t/ Q 5 (aO. (3.6) 
The following Theorem will be used extensively throughout this paper. 

Theorem 1. Let S = {St,t > 0} be a strong Markov process. Define, 
Q = {a(S u , u < t)} t>0 , t to be any stopping time with respect to Q . Let Y be 
the process obtained by killing the process S at time r, denote the state space 
of this process by E, and let U a be its a— potential. Then, for x £ E 

E x [e- aT ] = l-aU a l E {x). (3.7) 

Proof. From the definition of U a , for any bounded measurable function / 
whose domain is E, we have 

U a f(x) = E x [f e- at f(S t )dt}= [ f(y)U a (x,dy)- 
Jo Je 

Taking / to be identically equal to one, we have 

1 = U* lE{x) . 
a 

The required result is immediate from the last equation above. I 



First we consider the case where the input process is a spectrally positive 
Levy process. 

(i) 

Proposition 1. For a > 0, a < A the a-potential (U a ) of the process 

A 

I killed at T = To A T~ is absolutely continuous with respect to the Lebesgue 
measure on [a, A] and a version of its density is given by 

u°(x, y) = W^(X x) = ° } - W^(y x), x,y e[a,\]. (3.8) 
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Proof. For A C [a, A] 



U a (x,A) = E x [ e- at l {IteA} 
Jo 

-I 

Jo 



f T-.AT+ 

— A — a 



= E_ x I e- at I a dt 



= E X - X J e l {i ex A } dt 

where the last equation follows from Theorem 8.7 of [7], this establishes our 
assertion. ■ 

Corollary 1. For a > the a-potential (U a ) of the process X is absolutely 
continuous with respect to the Lebesgue measure on (-co, A] and a version of 
its density is given by 



u a (x,y) = W^ a \\-x)e-^ a ^ x -^ -W (a) (y-x), x, y e (-co, A]. (3.9) 

Proof. The proof follows from (3.8) by letting a — > — oo and since, for a > 0, 
W (a Hx)~ /," ( ° ) * asx^oo. ■ 

a A 

We are now in a position to find £^[e~ aTo ] and E x [Tq]. 
Proposition 2. (i) For a > and i<Awe have 

£ x [e-^°] = Z< a >(A-:c)--^W , < a >(A-aO. (3.10) 
(ii) For x < A we have 

£*P\>] = -W(X-x), V (0)>0 

= oo , ?7(0) = 0, (3.11) 

where for every x > 0, 

W(x) = f X W(y)dy. 
Jo 

Proof. We only prove (i), the proof of (ii) is easily obtained from (i) and 
hence is omitted. Let U a be as defined in Corollary 1, then 
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E x [e- aT °] = l-aU a l { ^ x) {x) 
r x 

= I- a {W a (X- xje-^-y^ -W a (y-x)}dy 

J —oo 

= l + a W a (y-x)dy-aW a (\-x) e'^-^^dy 

J x J — oo 

= Z {a \\-x)-^—W {a) (\-x) 1 
77(a) 

where the first equation follows from (3.7), the second equation follows from 
(3.9), the third equation follows since (x) — 0, x < 0, and the last equation 
follows from the definition of . ■ 

For any Borel set B C R+ x i?, we let M(B) be the Poisson random measure 
counting the number of jumps of the process I in B with Levy measure v, where 
if B = [0, t) x A, Ac R, then E[M(B)] = tv(A). 

(i) 

Proposition 3. Let u a be as given in (3.8), and x < A < z, then 

a A r\ (1) 

Eje~ aT «,I, edz, T <T-}= v(dz-y)u a (x,y)dy (3.12) 

T o J a 

Proof. Let T be as defined in Proposition 1. For x < A, a > 0, C C 

[A, oo) and D C (a, A) we have 

E x [e-°^° , Ja G C, 7a e D, T < T"] 

I o J- a — 



E x [ e- at I I {yeC -i t - } M(dt,dy)} 

J[0,oo)x(0,oo) {It-<\Jt->aJt-eD} 



= E x [f e- at I l {heD} v{C - I t )dt] 

J[0,oo) {It-<\,h_>a} 

= E x [[ e- at l {t<T} v{C - I t )I {IteD} dt)] 

J[0,oo) 

= E x [f e- at l {t<T}V (C-y)\ {hedy} dt] 

J[0,oo)xL> 



(1) 

v(C - y)u a (x,y)dy, 

D 

where the second equation follows from the compensation formula (Theorem 
4.4. of [7]). Our assertion is proved by taking D = [a, A]. ■ 

The following corollary gives a formula needed to compute the total dis- 
counted cost. 
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Corollary 2. Let u a be as defined in (3.9). For a > and for x < A < z, 

E x [e~ a ^\I, edz}= [ v{dz-y)u a {x,y)dy. (3.13) 

T o J-oo 

Proof. The proof follows immediately from (3.9) and (3.12) by letting a — > 
— oo. ■ 

We now turn our attention to the case where the input process is a spectrally 
positive Levy process reflected at its infimum. In this case, the killed process 

(2) 

has state space [0, A). Let U a be the a- potential of this process. 
Proposition 4. For any x,y € [0, A), 



( 2 ) VF (a) (A - x)W( a Hdy) , n 

U a (x,dy) = 1 J K V) WW(y - x)dy, (3.14) 
W[ a) (A) 

where for x,y e [0, A), W {a \dy) = (0)6 (dy) + W^ Y (y)dy, and S is the 
delta measure in zero. 

Proof. Note that for each t > 0, 

It = Yt-Yt (3.15) 

A A 

= Y t -Y t , 

where the process Y = {Y t ,t > 0} is a spectrally positive Levy process. The 

A 

result follows from part (ii) of Theorem 1 of [10], since the process Y is a 
spectrally negative Levy process. ■ 

The following provides results parallel to (3.10) and (3.11), resectively. 

Proposition 5. Assume that the input process is a spectrally positive Levy 
process reflected at its infimum. Then 

(i) For a > and x < A we have 

EJe- a ^°] = Z^\X - x) - W {a) {\ - x) aW f ^ X \ (3.16) 

(ii) For x < A we have 

E x [T ] = W(X-x)^- ) -W(X-x). (3.17) 

Proof. The proof of part (i) follows from (3.7) and (3.14), in a manner 
similar to the proof of (3.10). The proof of part (ii) follows from part (i) by 
direct differentiation. ■ 
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To find a formula analogous to (3.13), when the input is a spectrally positive 
Levy process reflected at its infimum, we first need few definitions. Define 



l a (dz) = W {a) {X-x) / W (a (dy)v(dz-y) 

Jo 
r x 

-W{ a) (\) / dyW (a \y-x)v(dz-y)],z > A. (3.18) 
Jo 

L a (z) = ( l a {du). (3.19) 

J (2,00) 

V a (A) = W^' {X)Z {a \X-x)-aW {a) {X-x)W^\X). (3.20) 

The following proposition gives the required formula. 
Proposition 6. (i) For a > and for x < A < z, 

E x [e"^° , J A € dz] = . (3.21) 

r J (A) 

(ii) For a > 

£7.[e-^" )/A =X]= Va(X) ( - La{X) . (3.22) 
^ VF[ a) (A) 

Proof, (i) Consider the spectrally positive Levy process Y — {Yt,t > 0}, 
given in the proof of Proposition 4. For any a g i?, we define F a as the sigma 
algebra generated by (Y s ,s < t), t+ = inf{t > : Y t > a}, = inf{£ > : 

Y t < a}, a+ = in£{t >0:Y t > a}, and a~ = inf{t > : Y t < a}. From 

A 

(3.15), for x > 0, Iq = x if and only if Y — x if and only if Y = —x . 
Furthermore, T = and I A = Y T + almost surely on {t\ < Tq }. Therefore 

To A 

E x [e- a ^ a ,I A g dz]=E x [e- a T°J A e dz,T+ < t-}+ E x [e~ a T°,I A edz,T+>T^} 

To T T 

= E x [e- a ^ ,Y Tt edz,T+<To] 

+E x [e~ aT ° ,r+ > tq] x E Q [e- a ^\I. g dz] 
= E x [e~ aT t ,Y rt edz,r+<r -] 



[e^ CT ° + , a! A > C7+] x E [e~ aT <> , J A g dz] 

To 

E.le-^t ,Y Tt edz,T+<To] 

+E x _ x [e- a °l , a " > <7+] x £ [ e "^° , I A g dz] , 
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where the second equation follows from the first equation by conditioning on 
F - and then using the strong Markov property. The third and fourth equations 



A 

r + 



follow from the definitions of Y, r^,T a ,o^,a a . 

Letting a — > in (3.8) and (3.12), we find that the first term in the last 

A 

equation above is equal to J v{dz — y)[W^(X — x) j^ Q )[^ — W^^y — x)]dy. 

o 

The second term is equal to W w (l^^ (see (8.8) of [7]) and the third term is 
equal to h "}y Z ^ (this follows from Theorem 4.1 of [11] by letting the (3, 7 — > 0). 

W + (A) 

Our assertion is satisfied by replacing each of the three terms in the last 
equation by the corresponding value indicated above and after some algebraic 
manipulations, which we omit. 

(ii) The proof is immediate from (3.16) and (3.21). ■ 



Now we turn our attention to computing C a (x,r 1 M), E x [exp(— aT )], and 

E X [T ], whenxe [X,V]. 

* 

Let rj T = inf{£ > : I t < t} and, for each t > 0, 
X t = {I t ,t<r] T }. (3.23) 

Note that, the state space of the process X is the interval (t, V], and let U be its 

a-potential. Starting at any x <G [A, V], r\ T = T almost everywhere, furthermore 
the sample paths of a spectrally positive Levy process and a spectrally positive 
Levy process reflected at its infimum behave the same way until they reach 

level t, thus X behaves the same way in both cases. It follows that, for each 
xe[X,V], 

C a (x,T,M) = ii a g*(x). (3.24) 

Denote the process I—M by N, note that this process is a spectrally positive 
Levy process with the Levy exponent </>m(^) = 4>{0) + 6M, 9 > 0. We denote 

its a— scale and adjoint a— scale functions by wffi and zff, respectively. 

* a 

Theorem 2. For a > 0, U is absolutely continuous with respect to the 
Lebsegue measure on (r, V], and a version of its density is given by 



u\ X ,y) = Z " iV l^» { "- T) -w£\y- X ) , x,ye(r,V}. (3.25) 

Z M ( V - T ) 
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Proof. For each t > 0, we define B t — N t — V. For any b G R, we define 

o-^ = inf{i > : B t - B t < b} and 7+ = inf{i > : B t - Bt > b}. For any Borcl 
set A C (r, V] and x G (r, U] we have 

P x {x t e 4} = P I {/ i ei,f<i) T } 

= P x {7V t - su P ((iV s - V) V 0) G A, t < Vt } 

8<t 

= P x - V {B t - B t G A - V, t < a~_ v } 

= P V - x {B t -Bt €V-A,t< 7^_ T } 
Using Theorem 1 (i) of [10], the result follows. ■ 

The following theorem gives Laplace transform of the distribution of the 
stopping time To and E x [Tq] when 16 [A, V]. 

Theorem 3. (i) Let x G [A, V] and a G R+, then 

E x [e- a *°]= Z ^ {V - X ] . (3.26) 



(ii) For x G [A, V] 
E x [T ] = Wm{V-t)-Wm{V-x), (3.27) 

X 

where, W M (i) = J W M {y)dy. 


Proof. We only prove (i), the proof of (ii) follows easily from (i) and is 
omitted. For x G [A, V], we have 



E x [e' aT "} = 1-aU I (T ,v](aO 
f v 

= 1 — a u (x, dy) 



- 1 rfcffM.^.* 



r^ } (y-») r Z<"(V - r) - 1 , r ^-i)-l. 

V(^-t) 



= i- a [;^\ :i ^ i, >-{ mv r >i 



Q 



g)" - - Z'm' (V-x) + Z%> (V x) 



J M 



Z${V-x) 



Z { m\V-t) 
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where the third equation follows from (3.25), the fourth equation follows from 
the definition of the function zffi a 
equation after obvious manipulations. 



the definition of the function zffi and the fifth equation follows the fourth 



Remark 1. When V = oo, for a > we let rj M (a) = sup{6> : (f)(9) — 6M = 

.(a) 
M 



a}. Since Z^\y) — 0(e VM ^ v ) as y — > oo, then we have 



»?m(«)'(0) 
(x-t) 



, if M > 



M-E(I{] 
= oo , if M < E{h). 

This is consistent with the well known fact about the busy period of the M/G/l 
queuing system. 

The following gives E x [ex.])[[—aT )}, when x < A, a result that is needed to 
compute the total discounted cost. 

Theorem 4. Assume that the input process is a spectrally positive Levy 
process. For z > A, we define 



h a (x,dz) = u a (x,y)v(dz-y)dy, 

J —OO 



where u a (x,y) is defined in (3.9). 
Then, for a > 0, x < A 



* i rV rOO 

E ^ e ~ aT °] = ^ [/ Z ( £\V-z)h a (x,dz)+ h a (x,dz)] (3.28) 

z m( V - t ) Jx Jv 

Proof. We write 

E x [e- aT "} = E x [e- a ^°- a(T °-^ o) } 

= E x [E x [e- a ^- a ^-^\a(T ,I.)] 

To 

= E x [e-^° E {lA AV) [e-^°]] 

To 



J M 

rV 



1 



Z$(V-ryJx 



[I Z ( A f{V - z)h a (x,dz) + I h a (x,dz)} 
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A * A 

where the third equation follows since, given T and J A , To — To is equal 

To 

to T almost everywhere. The fourth equation follows from (3.26). The last 
equation follows from (3.13), the fact that Z^\o) = 1, and the definition of 
h a (x 7 dz). ■ 

The following theorem gives a result analogous to (3.28) when the input 
process is a spectrally positive Levy process reflected at its infimum. 

Theorem 5. Assume that the input process is a spectrally positive Levy 
process reflected at its infimum. For z > A, let l a (dz), L a (z), and V a (\) be as 
defined in (3.18), (3.19) and (3.20), respectively. Define 

r = J0^, Z >\ 

g a {x,dz) = I _ V q (^)_ Lq(A) 
{ - <>'(A) 

Then, for a > 0, x < X 



E ^ aT °^^T; :[/ Z${\-z)g a (x,dz)+ g a (x,dz)}. (3.29) 

Z M ( X ~ T ) Jx Jv 

Proof. The proof follows in a manner similar to the proof of (3.28), using 
(3.21), (3.22) and (3.26). ■ 



4. The expected total discounted and long-run average 
costs 

Consider a finite dam controlled by a Pjf T policy as described in Section 
1. Assume that the input process, /, is spectrally positive Levy, and define a 
to be the discount factor. For x G we let C"(A,t), and C(A,r) be the 

expected total discounted cost and long-run average cost, respectively, given 
Iq = 0. Furthermore, we define C a (x) as the expected discounted cost during 

the interval [0,To), given the initial water content is equal to x. 
Modifying (3.1) of [1], it follows that for x G [r, V], 

C:{X,r) = C°{x) + ^^P . (4.1) 
l-£ T [exp(-aT )] 



From the definition of the P^ T policy, it follows that for A < x < V 
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C a {x) = M{K X - RE^ / e- at dt} + C a (x,T,M), (4.2) 
Jo 

and for x e [r, A] 

C„(aO = M{^ 2 + ATi E x [e- a S--{EAe- a S - E x [e~ a ^] } 

a 

+C a (x,X,0) + E x [e- a ^°C a ((h A V), t, M)], (4.3) 

To 

where C a (x, A, 0), and C a (x,T,M) are given in (3.6) and (3.24), respectively. 
Using (3.9), (3.10), (3.13), (3.25), (3.26) and (3.28) we obtain C a (x). Finally, 
the expected total discounted cost can be determined explicitly by substituting 
(4.2), (4.3), (3.26) and (3.28) into (4.1). 

To determine the long-run average cost using a given Pyf T policy, we proceed 
as follows. Let C(A,r) denote the long-run average cost, and define C°(x) as 

the expected non-discounted cost during the interval [0,To), given the initial 
water content is equal to x, x € [r, V}. It follows that 

C{\,t) = ^L. (4.4) 

E T [T a ] 

From the strong Markov property we have 

E T [T ] = E T [T ] + E t [E IJa AV) [To]]. (4.5) 

Furthermore 

C°(r) = M{K-R(E t [T o ]-E t [To])}+C o (t,X,0)+E t [C o ((I, AV),t,M)}, (4.6) 

T A 

where K = K± + • Letting a = in (3.13) and substituting the result, along 

with (3.11) and (3.27) into (4.5) we obtain E T [T ]. Using (3.6), (3.9), (3.11), 
(3.13), (3.24), (3.25) and (4.5) we obtain (4.6). Substituting (4.5) and (4.6) into 
(4.4) the long-run average cost is determined. 

The corresponding results for the spectrally positive Levy reflected at its 
infimum input follow similarly. 
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5. Special Cases 



In this section we consider the cases where the input process is a spectrally 
positive Levy of bounded variation, Brownian motion reflected at its inhmum 
and Wiener process. For the first case, we extend the results of [6], we also 
simplify some of their results. For the second case, we obtain results similar to 
those of [4] and [8]. In the third case we obtain the results of [12]. 

Case 1. Assume that the input is a spectrally positive Levy process of 
bounded variation with Levy exponent described in (2.3), reflected at its infi- 
mum. Let /i = J °° xv(dx) and assume that p < oo. For every x € R+, we define 
the probability density function f(x) — HHiEiEHll. We have / °°(1 — e~ 6x )v(dx) = 
9p J °° e~ 6x f(x)dx. Define p = ^, F(x) as the distribution function correspond- 

oo 

ing to / . Assume that p < 1, it follows that, ^ = ± J °° e~ 9x dx £ p n F (n \x). 
Therefore, the 0— scale function is given is given as follows 

W(x) = -^p n F^ n \x). (5.1) 

For a > 0, is computed using (2.5) and (5.1). 

Define £ = ? + M, and p = Let wffl be as defined in the paragrah 

proceeding Theorem 2, and denote by W M , using an argument similar to 
the one above we have 

1 oo 

? n=0 



It follows that, for a spectrally positive Levy process of bounded variation, 

' (2) 

W(0) — 0. Thus, the a-potential U a (given in (3.14) is absolutely continuous. 
From (4.4) (4.5), and (4.6), the long-run average cost is determined once for 

A * ( 2 ) ^0 

x e [A, V], E T [T ], E x [T ], U°, u , and the didtribution of I a are computed. 

To 

Using (5.1) and (3.17) we compute E T [T }, and using (5.2) and (3.27) E X [T ] is 

(2) 

determined for x € [A, V]. Furthermore, U° is computed using (3.14) and (5.1). 
From (3.25) it follows that, for x, y e (r < V],u°(x,y) = W M (y—T)—W M (y—x), 
which is determined using (5.2). The distribution of I A is given by letting a — > 

in (3.21) and (3.22), and using (3.18), (3.19), (3.20), and (5.1). 



16 



The corresponding results for the total discounted cost follow similarly. 
Remark 2. 

(a) Bac et al [6] obtain the long-run average cost, when the input pro- 
cess is a compound Poisson process with a negative drift. In this case, v(dx) = 
XG(dx), where A > and G is a distribution function of a positive random vari- 
able [0, oo), describing the size of each jump of the compound Poisson process. 

In this case, /(x) = G ^ and p = where G = 1 — G and m = J °° G(x)dx, 
which is assumed to be finite. We note that their entities w(x) and E[L a (X,r)} 
given in p. 521 and p. 523 of [6], respectively, are nothing but our Co(x, A, 0) 

A 

and E t [Tq], respectively. Furthermore, for x G [r, V], their functions u(x) and 

E[T^ (x)] given in page 524 are our Co(x, r, M) and E X [T ], respectively. The 
distribution of L(t) (the overshoot) given on page 525 follows in an obvious 
manner from the distribution of I a . The formulas for computing Co(x, A,0), 

To 

A * 

E t [Tq], Cq(x, t, M), E x [Tq], and the distribution of 7a , follow from the corre- 

To 

sponding results obtained in Case 1. We note that our formulas for computing 
w(x) and E{L a (\,T)} are identical to those of [6], while our formulas for u(x), 
E[T^ (x)], and the distribution of L(t) are simpler than theirs. 

(b) Assume that the input process is a gamma process with negative drift. 
The Levy measure is given by v(dx) — a^—dx^ a, b > 0. In this case, E(Ii) = 
<T — | , which is assumed to be nonnegative and p = ^ < 1. It follows that 

f(x) = b e b y dy, the right hand side is denoted by E\(x) in p. 227 of [3], 

Direct integrations yield, F(x) = (1 — er hx ) + xf(x). 

(c) Assume that the input process is an inverse Gaussian process with a 
negative drift, and with Levy measure is given by v{dx) = —j=== e~ xc / 2cr , 
a,c > 0. It follows that E(I\) = <; — i, which is assumed to be greater than 
zero. In this case p = < 1, f(x) = c v(dy), and F(x) = cii(cy / y/2a' 2 ) + 
xf(x*j . 

(d) Since, for all n > 0, F^(x) < [F{x)] n , then for all x, W{x) < ^^ F{x) , 
if p< 1. 

Case 2. Assume that the input process is a Brownian motion with drift 
term p G R, variance term ct 2 , reflected at its infimum. From (3.6), (3.24), and 
(4.1)-(4.6), the total discounted and long-run average costs are determined once 

E x [e- a ^°], E x [e~ aTo ], E T [T a ], E X [T ], U", and u* are computed. 

In this case, I a — A almost everywhere, the Levy measure v = 0, and 

To 

from (2.1) we have, for 9 > 0, <j>{6) = -p6 + It follows that, for 

a > 0, r](a) = \/^Z+Z±^ , Let S = y/2a<r 2 + p 2 , we have, W^(x) = 
2 e^ x ^ 2 sinh(ff ) and Z^ a \x) = e^l^ (cosh(^) - f sinh(^)). We note that 
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W a (x) is differentiable, and W^' (x) = £W^(x) + ^e^l^ cosh(f| ); hence 

. Substituting the values of Z^(X - x), W {a \\ - x) 



vy (o) (A) 



W<°)'(A) 



in (3.16), we have, for a > 0, x < A 



-aT i _ e( u(A-x) 



cosh ((^M)_i sinh (^_ 



x)<5 



it + <5coth(£f) 



a simpler and more explicit formula than (4.6) of [4]. 

(2) 

We note that, W^ a \0) = 0, U a is absolutely continuous with respect to the 
Lebesgue measure on [0,A). Substituting the values of W^(X - x), W a '(X), 

(2) 

W( a \y — x), and W a (y) in (3.14) we get a version of the the density of U a . 



Let Ji = n-M, S = V 2aa 2 + ft , it follows that W^(x) 



smh(f|), 

and z["\x) = e^/"" 2 ^cosh(^|) - |sinh(^§)]. Hence, u is computed using 
(3.25). Since J A = A almost everywhere, 



^[e- Qj °] -is x [e- Qj °]is A [e-^°], (5.4) 

where E x [ e - a ^°] is given in (5.3) and E x [er aTa ] = Z f a ] ( ^~ X) , which follows 
from (3.26). 

A * 

To compute E t [Tq], E\[Tq], we first assume that /x#0. In this case, for 



x > 0, W(x) = * * x 'l -\ W'(x) = an d W(x) 



2)j.' 



2fix/cr 2 



Substituting the values of W{\ - x), W(X - x), W(A) and W (A) in (3.17) we 
have, 



E T [T ] 



A 



/z 2/i 2 

Since, /a = A almost everywhere, from (4.5) we have 



(5.5) 



£7 T [T ] =£MTo]+£a[To]. (5.6) 



We note that Wm is computed from Vy above by replacing the term fi by 
[i defined in the preceding paragraph. Let A* = V — A, and r* = V — r, 

substituting the values Wm(V — r) and Wm(V — A) in (3.27) we have 
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Ex[T ] 



X* 



2/j, 



■ (5-7) 



If /i = 0, then W{x) = p and W{x) = ^. From (3.17) it follows that 



E T [T ] 



X 2 - X 2 



(5.8) 



It is easily shown that, W M {x) = jj{ 1 - e^ 2 ^/* 72 ), using (3.27) we have 

r 2 



t* — A* a 
+ 



M 



2M 2 



e -2Mr'/a 2 _ g -2MA*/c 



. (5.9) 



We note that our (5.5) is consistent with (4.9) of [4], while (5.8) is identical 
to the corresponding equation given in p. 298 of the same reference. 

Case 3. Assume that the input process is a Brownian motion with drift term 
[i > and variance parameter a 2 . It follows that 77(0) = Substitut- 
ing the values of W^ a \x), Z^(x), given in Case 2, in (3.10) we have, for 

x < A, E x [e-°^°] = exp ((5 — [i)(x — A)). Substituting ^(e 2 ^ 2 - 1) and 

^.( e 2W°" 2 _ 1) _ s. f or W(x) and W(x), respectively, in (3.11), we have, for 

a 

x < A, E x [Tq] = These results are consistent with the results of Zucker- 

man [12], p.423. The values of E x [e^ aT °] and E\[T } are the same whether the 
input process is spectrally positive Levy or spectrally positive Levy reflected at 
its infimum. The computations of the total discounted and long-run average 
costs can be obtained using (3.9), (3.13), (3.25), (4.1)-(4.6) and the values of 

A * A * 

E x [e~ aTa ], E\{e~ aT °] 7 E X [T ], E\[T ], in manners similar to those discussed in 
Case 2. 
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